A fast randomized algorithm for the approximation of matrices — preliminary report ∗

Given an m × n matrix A and a positive integer k, we introduce a randomized procedure for the approximation of A with a matrix Z of rank k. The procedure relies on applying an l×m random matrix with special structure to each column of A, where l is an integer near to, but greater than k. The spectral norm ‖A−Z‖ of the discrepancy between A and Z is of the same order as √ lm times the (k + 1)st greatest singular value σk+1 of A, with small probability of large deviations. The special structure of the l ×m random matrix allows us to apply it to an arbitrary m × 1 vector at a cost proportional to m log(l). Utilizing this special structure, the algorithm constructs the rank-k approximation Z from the entries of A at a cost proportional to mn log(k) + l2 (m + n). In contrast, the classical pivoted “QR” decomposition algorithms such as Gram-Schmidt cost at least kmn. If l is significantly less than m and n, then the randomized algorithm tends to cost less than the classical algorithms; moreover, the constant of proportionality in the cost of the randomized algorithm appears to be small enough so that the randomized algorithm is at least as efficient as the classical algorithms even when k is quite small. Thus, given a matrix A of limited numerical rank, the scheme provides an efficient means of computing an accurate approximation to the singular value decomposition of A. Furthermore, the algorithm presented here operates reliably independently of the structure of the matrix A. The results are illustrated via several numerical examples.